Image Processing Apparatus And Method

ABSTRACT

An image processing apparatus ( 400 ) comprises a SIMD processor ( 401 ) which scans an image frame for regions of interest (step  301 ), for example corresponding to regions having objects or lines of interest. Each region of interest is rescanned to an orthogonal grid. The orthogonal grids are then floorplanned so that they are rearranged into a smaller subset of image lines. The floorplanning consists of mapping a set of rectangles into a compressed frame portion. Optionally, the rectangles can be rotated in order to allow the rectangles to be packed more densely. The SIMD processor ( 401 ) then processes the floorplanned image data (step  307 ). Once the image data has been processed by the SIMD processor, the DSP ( 405 ) re-associates the processed data (step  309 ), using information stored during floorplanning. The image processing apparatus results in a more efficient use of the SIMD processor ( 401 ).

The invention relates to an image processing apparatus and method, andin particular, to an image processing apparatus using Single InstructionMultiple Data (SIMD), in which floorplanning of SIMD tasks is employedto provide more efficient SIMD processing.

SIMD processing is a powerful computing paradigm for applications thatexhibit massive parallelism. One such application that adopts the use ofSIMD processing is that of image processing. SIMD processors, forexample Xetal, perform their operations on each data item (e.g. eachpixel in a line for Xetal) whether they are needed or not. In otherwords, a processing operation is performed on a pixel in a lineregardless of whether or not a processing operation is required.Depending on the data distribution or sparsity, much computation powercan therefore be wasted using this technique.

More and more image processing algorithms are being developed to work onportions of images. For example, in television processing, industrialvision or medical imaging, it is known to work on the edges of images(i.e. line processing). Also, in applications such as imagecommunication or 3 D rendering, it is known to work on separate objectswithin an image (i.e. object processing), thereby reducing the amount ofunnecessary processing operations.

Several solutions exist for making efficient use of SIMD computingresources. For example, one method is to load-balance over multiple SIMDprocessors. Another is to provide algorithms that use special datastructures to operate efficiently on sparse structures. For example,such a technique is disclosed in “Massive parallelism for sparseimages”, Shankar et al, IEEE International Conference on Decision Aidingfor Complex Systems, 1991. However, such systems suffer from thedisadvantage that they require control and hardware overheads.

The methods described above also suffer from the disadvantage ofprocessing data items which are of no interest.

The aim of the present invention is to provide an improved imageprocessing apparatus and method which does not suffer from thedisadvantages mentioned above, and in which the number of unnecessarydata operations is reduced.

According to a first aspect of the present invention, there is providedan image processing apparatus comprising a processing means adapted toreceive an image signal and identify regions of interest within an imageframe. A rescanning means is adapted to rescan each region of interestinto an orthogonal grid. The rescanned regions are then rearranged byrearranging means into a compressed frame portion, such that theprocessing apparatus processes the rearranged regions of the compressedframe portion.

The invention has the advantage of only processing the compressed frameportion, thereby making more efficient use of the processing apparatus.

According to another aspect of the present invention, there is provideda method of processing an image signal using a SIMD processor. Themethod comprises the steps of identifying regions of interest in animage frame, and rescanning each region of interest into an orthogonalgrid. The rescanned regions are then rearranged into a compressed frameportion, such that only the compressed frame portion is processed by theSIMD processor.

For a better understanding of the present invention, and to show moreclearly how it may be carried into effect, reference will now be made,by way of example, to the accompanying drawings, in which:

FIG. 1 shows an image having objects sparsely distributed within animage frame;

FIG. 2 shows the result of floorplanning the objects of FIG. 1 prior toprocessing, in accordance with the present invention;

FIG. 3 shows the steps involved in the floorplanning operation;

FIG. 4 shows the mapping of tasks to a vision architecture; and

FIGS. 5 a and 5 b show how a line or edge may be reshaped prior toprocessing.

FIG. 1 shows an image frame 1 comprising a plurality of objects 3. ASIMD processor working on the image frame 1 identifies the regions ofinterest within the image frame 1. The regions of interest correspond,for example, to the regions where the objects 3 are located.

After identifying a region of interest, for example by the SIMDprocessor, the region of interest is rescanned to an orthogonal grid 5,for example using the techniques described in co-pending patentapplication ID612814. The rescanning process involves rescanning regionsof an image to line or rectangle based regions on which a SIMD processorcan efficiently perform its line or rectangle based processing.Preferably, the rescanning of the region of interest onto an orthogonalgrid is done to place a line or an edge onto a column or row. However,it is not essential that this is done exactly on a row or column, sincethis would be impracticable.

Since a region of interest having an object 3 is rescanned to anorthogonal grid 5, the amount of further processing required by the SIMDprocessor is reduced, and is limited to the lines that fall together onthe shortest dimension of the orthogonal grid 5.

Although the arrangement shown in FIG. 1 might slightly reduce thenumber of computational operations performed by the SIMD processor, itstill performs a number of unnecessary operations on all image partswhere there are no objects.

FIG. 2 shows the image processing operations performed in accordancewith the invention. As described in FIG. 1, a pre-processing operationis performed to identify the regions of interest where the objects 3 arelocated. Each region of interest is then rescanned to an orthogonal grid5. However, prior to processing the image data, the orthogonal grids 5corresponding to the regions of interest are floorplanned into acompressed frame portion 7.

This means that the further processing only has to be performed on asubset of the lines in the image frame, corresponding to the compressedframe portion 7. Additionally, since the subset of lines in thecompressed frame portion 7 are packed more densely with regions ofinterest, more efficient use of the SIMD processor is achieved.

FIG. 3 describes in greater detail the steps performed according to theimage processing method of the present invention. In step 301, theregions of interest are identified within an image frame. The regions ofinterest correspond, for example, to regions having objects 3 ofinterest. In step 303, each region of interest is rescanned to anorthogonal grid.

Then, in step 305, the orthogonal grids are floorplanned so that theyare rearranged into a smaller subset of image lines, corresponding to acompressed frame portion. The floorplanning step 305 consists of mappinga set of rectangles, i.e. orthogonal grids 5, into a compressed frameportion 7. Optionally, the rectangles can be rotated in order to allowthe orthogonal grids to be packed more densely into the compressed frameportion 7. Preferably, the floorplanning step is performed using ageneral purpose processor that is used to assist the SIMD processor. Incontrast with conventional floorplanning algorithms used for otherpurposes, the floorplanning operation performed by the present inventionstores information relating to the movement (and possibly informationrelating to the rotation of) the original rectangles, for later use asdescribed below.

The SIMD processor then processes the floorplanned image data, step 307.Since the SIMD processor performs a similar instruction for all pixelsin a row, the floorplanned image data is processed more efficiently.This is because more objects are packed on a row, which means that morepixels are usefully processed. Once the image data has been processed bythe SIMD processor, the results are re-associated in step 309 to theiroriginal frame positions, using the stored information mentioned above.This involves re-associating the computed data with the regions of theimage prior to the floorplanning operation.

Optionally, the rescanning, floorplanning and SIMD processing steps 303,305, 307 can be re-iterated if needed (step 311) until the desired levelof processing has been reached.

FIG. 4 shows a preferred embodiment describing how the steps performedin FIG. 3 are realized in the image processing apparatus. The imageprocessing apparatus 400 comprises a memory 407 and a display processor409 for providing image data 411 to a display device (not shown). Theimage processing apparatus 400 comprises a SIMD processor 401 whichreceives input image data 402 from a sensor (not shown). The SIMDprocessor 401 is used to identify the regions of interest within areceived image signal (i.e. corresponding to step 301). Data from theSIMD processor is processed by an FPGA 403, which rescans the image datato an orthogonal grid, corresponding to step 303. As mentioned above,the floorplanning operation, step 305, is preferably performed by ageneral purpose processor, for example a TriMedia DSP 405. Thefloorplanned image data is then processed by the SIMD processor 401,with the re-association or re-mapping (step 309) being performed by theTriMedia DSP 405.

The invention described above provides an image processing apparatus andmethod in which more efficient use of SIMD processing is provided.

It will be appreciated that the invention is not limited to the specificarchitecture described in the preferred embodiment, and other hardwarearchitectures could be used to provide similar functions to thosedescribed above.

In addition, although the preferred embodiment relates to identifyingobjects of interest in the image, the invention can equally be appliedto lines or edges of interest, which are rescanned to an orthogonalgrid. For example, FIG. 5 a shows an image frame 501 having an edge 503.According to the invention, the edge 503 may be reshaped such that theedge lies within a reduced set of lines “N”, as shown in FIG. 5 b. Thereshaping information is stored, such that the image data processed bythe SIMD processor can be re-transformed to its original shape afterprocessing.

The invention can be applied to a number of different applications,including: the processing of television images to increase the imagequality; performing object recognition in computer vision applications;performing image rendering for computer gaming, education or CAD/CAM;performing object based coding for MPEG4, H263+; performing imageprocessing for medical systems.

It should be noted that the above-mentioned embodiment illustratesrather than limits the invention, and that those skilled in the art willbe capable of designing many alternative embodiments without departingfrom the scope of the invention as defined by the appended claims. Inthe claims, any reference signs placed in parentheses shall not beconstrued as limiting the claims. The word “comprising” and “comprises”,and the like, does not exclude the presence of elements or steps otherthan those listed in any claim or the specification as a whole. Thesingular reference of an element does not exclude the plural referenceof such elements and vice-versa. The invention may be implemented bymeans of hardware comprising several distinct elements, and by means ofa suitably programmed computer. In a claim enumerating several means,several of these means may be embodied by one and the same item ofhardware. The mere fact that certain measures are recited in mutuallydifferent dependent claims does not indicate that a combination of thesemeasures cannot be used to advantage.

1. A method of processing an image signal using a SIMD processor, themethod comprising the steps of: identifying regions of interest in animage frame; rescanning each region of interest into an orthogonal grid;rearranging the rescanned regions into a compressed frame portion; andprocessing the compressed frame portion in the SIMD processor.
 2. Amethod as claimed in claim 1, wherein the step of rearranging comprisesthe step of floorplanning the regions of interest into the compressedframe portion.
 3. A method as claimed in claim 1, wherein the step ofrearranging further comprises the step of rotating one or more regionsof interest, thereby enabling the area of the compressed frame portionto be reduced.
 4. A method as claimed in claim 1, wherein the step ofrearranging further comprises the step of storing information relatingto the original position of the region in the image frame.
 5. A methodas claimed in claim 3, wherein the rearranging step further comprisesthe step of storing information relating to the rotation of the region.6. A method as claimed in claim 4, further comprising the step ofre-mapping the regions after the processing step using the storedinformation.
 7. A method as claimed in claim 1, wherein a region ofinterest is one of a rectangle, line or object.
 8. A method as claimedin claim 1, wherein the step of rearranging is performed by a processorthat is separate from the SIMD processor.
 9. A method as claimed inclaim 1, wherein the rescanning, rearranging and processing steps arereiterated.
 10. A method as claimed in claim 1, wherein the rescanningstep further comprises the step of reshaping a line or edge in the imagesignal.
 11. An image processing apparatus comprising: processing meansadapted to receive an image signal and identify regions of interestwithin an image frame; rescanning means adapted to rescan each region ofinterest into an orthogonal grid; rearranging means adapted to rearrangethe rescanned regions into a compressed frame portion; and processingmeans for processing the rearranged regions of the compressed frameportion.
 12. An image processing apparatus as claimed in claim 11,wherein the rearranging means comprises floorplanning means forrearranging the regions of interest into the compressed frame portion.13. An image processing apparatus as claimed in claim 11, wherein therearranging means is adapted to rotate one or more regions of interest,thereby enabling the area of the compressed frame portion to be reduced.14. An image processing apparatus as claimed in claim 11, wherein therearranging means is adapted to store information relating to theoriginal position of the region in the image frame.
 15. An imageprocessing apparatus as claimed in claim 14, wherein the rearrangingmeans is adapted to store information relating to the rotation of theregion.
 16. An image processing apparatus as claimed in claim 14,further comprising means for re-mapping the regions after processing bythe processing means, using the stored information.
 17. An imageprocessing apparatus claimed in claim 11, wherein a region of interestincludes a rectangle or line.
 18. An image processing apparatus asclaimed in claim 11, wherein the rearranging means comprises a processorthat is separate from the SIMD processor.
 19. An image processingapparatus as claimed in claim 11, wherein the rescanning means,rearranging means and processing means are adapted to perform aniteration process.
 20. An image processing apparatus as claimed in claim11, wherein the rescanning means is further adapted to reshape a line oredge in the image signal.